Hi guys,
What did he do this time?
I made a pretty thorough analysis of input lag in the following sims:
- GPL
- LFS
- N2003
- Nkpro
- RBR
- Rfactor
How?
Using my Canon S3 IS digital camera to record, at the same time, the monitor image and a small joystick on a gamepad, engaged with a 'ghetto' spring loaded device. So I am recording the input (joystick) and the response (monitor) at 60 frames per second. I count the amount of frames it takes for the monitor image to respond to the joystick. I do this five times for each sim (at various framerates) and take the average.
Here is the 'plan' in a picture:
How accurate?
It is not highly academic. The monitor is a TFT running at 60Hz. So even with games running at 200fps, the monitor won't show all 200. Furthermore there will be some delay before the monitor responses. The camera at 60fps will probably not be in perfect sync with the TFT monitor. Still, roughly 1/60th of a second is a good accuracy and the results where consistent and there was enough difference between the sims and settings to give some interesting results!
What settings?
Various frame rate ranges (after tweaking graphics / underclocking PC to get the result within the range):
- Low framerates between 27 and 37
- Good framerates between 54 and 61fps
- Very high framerates between 90 and 140
- Insane framerates of about 230
System:
- AMD64 at 2.5ghz (3800+ abouts) underclocked for some of the low fps settings
- Nvidia GF6800GT 256MB Driver version 84.26, with AA / AF to reach target fps at times
- 1GB ddr at ~225mhz
- Windows XP sp2, full updates
- no 'usb overclocking' (standard 125hz speed. Can't yet be arsed to test this)
- Philips 20'' TFT rated at 30ms for what its worth
- Thrustmaster dual analog gamepad
Also:
- VSYNC on, 60 frames per second
- In LFS and NKpro, setting a max framerate in game (37Nkpro, 30 and 18 LFS)
Results
Compiled in Excel, the absolute delay (in seconds) versus the framerates, not limited by game limits or VSYNC are:
The relative lag, measured in simulation frames is:
How in game framerate limits (cap) and VSYNC affect the absolute input lag:
Conclusions
When not using VSYNC or in game frame rate limits, looking at the absolute lag:
- GPL makes a good show: The least amount of lag at the lowest framerate
- LFS is the best of the rest although it needs more than 100fps to get to GPL levels of response
- Rfactor is affected the most: at 27fps the lag is almost 0.25 seconds!
- RBR and NKpro aren't much better with terrible lag at lower framerates
- RBR and Rfactor need 200fps to be on a par with LFS at 100fps
When looking at how many of simulation frames the lag is:
- RFactor and RBR displays a linear ratio: double the framerate, halfs the input lag
- NKpro is unaffected, the lag is almost constant regardless of FPS
- LFS and N2003 do relatively better when the framerate is low
- GPL makes a great show again: it takes less frames AND it happens at a higher framerate (both have a positive influence on lag) to respond.
When using VSYNC or in game framerate caps:
- Vsync results are very close, between 0.1 and 0.115 seconds, except for GPL which is probably not working well with vsync as 60hz monitors and 36fps might not match..
- Framerate caps as selectable in LFS and NKpro work well
- NKpro 37fps, at 0.08 seconds is almost 2.5 times faster than when system performance limites the framerate at 37, with a whopping 0.19 seconds delay!
- LFS with a 30fps framerate cap comes in at close to 2.5 times less lag than running 'system limited' at 28fps.
Other things that struck me:
I have also looked at the number of video frames it took for the steering wheel to go from center to max lock. This was pretty constant at 3..4 video frames (making that 3.5 gives 3.5/60 = 0.058 seconds) Only N2003 was slightly slower at 4..5 frames.
In N2003 I seem to feel more lag than the numbers show. I do have a possible explanation. With all the other sims, as soon as the steering wheel moved, it showed a fair amount of movement. With N2003, very often, the first video frame showing wheel movement only showed very little movement. This seems to indicate that N2003 does not havea totally linear 'response speed'; as if there is some speed buildup.
What do I think?
Input lag is a bad thing. However, even you don't or barely see it, it is still there. The quicker a sim responds to your inputs, the more direct control you have over the car. In my opinion, there is NO reason that new games shouldn't be able to deliver the same quick response as GPL; a game from 1998!
I don't want to go deeply into a "when is it good enough?" debate. If technically a lag of 0.05 seconds is possible with a sim running at just 36fps, then this is the benchmark. A sim maker has no excuse when his sim has more lag at lower framerates. Anything has to be done to make simracing as 'immediate' as possible and reducing input lag should be very hign on the list.
Meanwhile, VSYNC doesn't help, which is sad as that looks so much nicer on TFT screens. It is obvious that, especially with Rfactor, RBR and NKpro, you are much better off sacrificing detail for speed.
Isn't it dubious that an 8 year old sim basically wins this? I can't call it a victory for RBR / Rfactor as they need to run at 6 times the framerate to beat GPL.. LFS is the best of the rest, especially at lower framerates.
Food for thought, and for the sim Devs out there! (hopefully)
Hope you enjoyed this
Niels